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ABSTRACT 

Background: Clinical trials on Traditional Chinese 
Medicine (TCM) should be registered in a publicly 
accessible international trial register and report on all 
outcomes. We systematically assessed and evaluated 
TCM trials in registries with their subsequent publications. 
Objective: To describe the characteristics of TCM trials, 
estimate bias risk and outcome-reporting bias in clinical 
trials. 

Data sources and study selection: Fifteen trial 
registries were searched from their inception to July 2012 
to identify randomised trials on TCM including Chinese 
herbs, acupuncture and/or moxibustion, cupping, tuina, 
qigong, etc. 

Data extraction: We extracted data including TCM 
specialty and treated disease/conditions from the 
registries and searched for subsequent publications in 
PubMed and Chinese databases. We compared 
information in the registries of completed trials with any 
publications focusing on study design, sample size, 
randomisation, bias risk including reporting bias from the 
register protocol. 

Results: 1096 registered randomised trials were 
identified evaluating TCM, of which 505 were completed 
studies (46.1%). The most frequent conditions were pain 
(13.3%), musculoskeletal (11.7%), nervous (8.7%), 
digestive (7.1%), circulatory (6.5%), respiratory (6.3%), 
mental and behavioural disorders (6.2%) and cancer 
(6.0%). The trial register data identified parallel, phase 
ll/lll randomised trials with sample size estimations and 
blinding, but limited information about randomisation 
(sequence generation and allocation concealment). 
Comparing trial registration data of 1 1 5 completed trials 
(22.8%) with their subsequent 136 publications, 
inconsistencies were identified in one or more of the 
following: sample size (11%), outcome assessor blinding 
(37.5%), primary outcomes (29%) and safety (28%) 
reporting. 

Conclusions: Increasing numbers of clinical trials 
investigating a variety of TCM interventions have been 
registered in international trial registries. The study design 
of registered TCM trials has improved in estimating 



ARTICLE SUMMARY 



Article focus 

■ We wished to evaluate the methodological 
quality of clinical trials in Traditional Chinese 
Medicine (TCM). 

■ We investigated whether the systematic identifi- 
cation of prior trial registration was associated 
with an improvement in the methodological 
quality of the subsequent published studies. 

Key messages 

■ A substantial number of clinical trials in TCM, cov- 
ering a broad range of therapies, are now being 
registered in international trial registries. 

■ Registration is associated with more rigorous 
study methodology and study design (eg, random- 
isation protocols, secure blinding and sample size 
estimates). 

■ Outcome-reporting bias exists when comparing 
the registry information and the subsequent pub- 
lications, and some trials were registered after 
their publication. 

Strengths and limitations of this study 

■ Systematic searches of all available international 
trial registries for any clinical trials of TCM. 

■ All interventions involving any TCM were 
included as was the diagnosis. 

■ The registered information for clinical trials is 
not uniform across the registries and important 
methodological information may be missing. 

■ Subsequent publications were obtained for those 
studies recorded as 'completed' in the registry. This 
may not represent the true situation for trials if the 
registry data are not updated by the researchers. 



sample size, use of blinding and placebos. However, 
selective outcome reporting is widespread and similar to 
conventional medicine and therefore study conclusions 
should be interpreted with caution. 
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Trial registration of Chinese medicine and publications 



Many empirical studies have shown that the methodo- 
logical quality of randomised clinical trials of Traditional 
Chinese Medicine (TCM) is poor with respect to risk of 
systematic errors (bias; generation of allocation sequence, 
allocation concealment, blinding, descriptions of drop-out 
or losses to follow-up, selective outcome reporting) and 
risk of random errors ( play of chance) . 1_6 Moreover, publi- 
cations on TCM trials are uniformly positive, raising con- 
cerns that trials initiated to investigate TCM are only 
published if they have positive results. 7 Poor-quality trials 
and risk of publication bias will reduce the strength of evi- 
dence when developing clinical practice guidelines or pre- 
paring systematic reviews. One of the ways to improve trial 
quality is to prospectively register clinical trial protocols in 
international trial registries such as clinicaltrials.gov. 8 9 
The WHO established an international clinical trial regis- 
try platform (ICTRP; http://www.who.int/ictrp/en/) in 
2005, which now links 14 clinical trial registries. 
Furthermore, several peer-reviewed journals such as The 
Lancet and Trials publish trial protocols to promote trans- 
parency and improve trial quality. 

In order to describe the characteristics of TCM trials, 
and estimate reporting bias in clinical trials, we systemat- 
ically searched 15 major international trial registries to 
identify information about TCM trials, and compared 
the registered records with subsequent publications 
regarding outcomes and other data. 

METHODS 
Inclusion criteria 

We included randomised clinical trials for any TCM 
intervention singly or their combination: Chinese 
herbs, acupuncture, acupressure, moxibustion, cupping, 
dietary advice, tuina (therapeutic massage), taichi, qigong 
and guasha (scaping massage). We excluded non- 
randomised studies such as quasirandomised studies, 
cohort studies, phase 1 trials, retrospective clinical studies, 
case series or case studies. There were no limitations on 
study type (superiority, non-inferiority or equivalence) or 
study phase. 

Data source 

We systematically searched 15 major international trial 
registries (14 linked to WHO ICTRP) from their incep- 
tion to July 2012 (see online supplement 01). 

For trials listed as 'completed' in the registered records, 
we then searched for published protocols, as well as the 
full texts of subsequendy published articles in PubMed 
and three Chinese electronic bibliographic databases 
including Chinese Biomedical Database (http://sinomed. 
imicams.ac.cn/index.jsp), China National Knowledge 
Infrastructure (http://www.cnki.net) and Chinese VIP 
Information (http://vip.hbdlib.cn/index.asp) . 

Data extraction 

Two researchers (from MH, X-XL, YJM, Y-YW or G-YY) 
extracted data independently from each trial registry 



using a standardised, piloted data extraction form. The 
form was developed by our research group and based on 
general characteristics of clinical trials, methodology 
and the 20 minimum items required for WHO trial 
registration. 10 The main information collected included 
the number of trials in each registry, year registered, 
trial design, methods, sample size, setting, participants 
and diseases/conditions, differentiation of syndrome 
(bian zheng lun zhi) , interventions, controls, primary 
and secondary outcomes, inclusions and exclusion cri- 
teria, current status (eg, completed and ongoing), 
ethical approval, sponsors, institutions, country of origin, 
contact details and funding. 11 The extracted data were 
cross checked by the authors, and any discrepancy 
resolved by discussion with a third author (J-PL). 
Conditions were classified according to the WHO inter- 
national classification of diseases (http://apps.who.int/ 
classifications/icdlO/browse/2010/en; last accessed 31 
July 2012). We searched for publications in bibliographic 
databases for those trials listed as 'completed' in the 
registries, and compared the published trials with the 
registered records. 

Data analyses 

We used The Cochrane Collaboration risk of bias tool to 
evaluate the registered records. 4-6 This tool assesses the 
following domains: generation of allocation sequence; allo- 
cation concealment; blinding; incomplete outcome 
reporting; and selective outcome reporting (defined as 
change of primary outcome or new outcome added) . We 
also evaluated the estimated sample size, explicit inclusion 
and exclusion criteria and the risk of funding bias. 

Two authors (MH and X-XL) compared the trial design 
and methodology from the registered records with the 
resulting publications to analyse the consistency and select- 
ive outcome reporting. Selective outcome reporting was 
defined as "when the full paper publications reported dif- 
ferent primary outcomes or changed it from the original 
pre-defined primary outcome in the registered data." 
Inconsistency was defined as "when the items were not the 
same as described in the registry in the subsequent paper 
publications." For the sample size estimation, a discrep- 
ancy of over 20% between the registered and published 
information was judged as inconsistent. We judged trials as 
positive results based on (1) authors' conclusion showing 
that the intervention was superior to the control; or (2) 
the comparison of between groups which showed statistic- 
ally significant differences ( p<0.05) for primary outcome 
measures. We compared the dates of registration and full 
paper publication to assess the proportion of retrospective 
registrations. 12 

RESULTS 

We identified 1096 registered records of randomised 
clinical trials on TCM. Five hundred and five of the 
1096 (46.1%) registered records indicated that the trial 
had been completed. Three hundred and thirty-seven of 
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Figure 1 Flow diagram of 
included trials in this study. 



WHOIntemationai Clinical Trials Registry Platform (ICTRP) 
n=14 Registries 



Clinicaltrials.gov 
n=l Registry 



Protocols n =1620 






Duplicates, non-TCM, CCT, cohort 
study, case-control, case series, etc 
n=524 








RCT protocols n=1096 










Data extraction 





Uncompleted n=591 



Completed n=505 



Data analysis 



Searching PUBMED, CNKI, VIP, CBM 



Publications 
n =136 (115 Trials) 



Data extraction 



Data analysis 



Abbreviation: 

RCT: Randomised control trial 

CCT: Controlled clinical trial 

TCM: Traditional Chinese medicine 

CNKI: China National Knowledge Infrastructure 

VIP: VIP Database 

CBM: Chinese Biomedical Database 



1096 (30.7%) were from mainland China (excluding 
Hong Kong and Macau) (figure 1). The first trial was 
registered in 1999, in clinicaltrials.gov, and the number 
of registered trials has increased over the past decade 
(figure 2). 



Table 1 shows the frequency of the type of TCM inter- 
vention included in each registry. The combined therap- 
ies studied included 105 trials on integrating Chinese 
and Western medicine, and 38 trials combining two or 
more TCM therapies. Using the international disease 



Figure 2 Number of registered 



randomised trials on Traditional 
Chinese Medicine by registry. 




1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 
Notes: 

1. Data in 2012 was not complete (Jan to July) 

2. There were no TCM trials in the Indian, Cuban, Pan-African, and Sri Lankan registries. 
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classification (ICD-10), we identified the 10 most fre- 
quent conditions: pain (13.3%) and musculoskeletal 
(11.7%), nervous (8.7%), digestive (7.1%), circulatory 
system (6.5%), respiratory conditions (6.3%), mental 
and behavioural disorders (6.2%), as well as cancer 
(6%), endocrine, nutritional and metabolic diseases 
(5.7%) and pregnancy and childbirth (4.6%; table 2). 

There were methodological variations across TCM inter- 
ventions (table 3). Our analysis of 1096 registered records 
showed that the majority were phase II/III (332/474, 
70%) with 1024 (93.4%) using parallel groups design. 
One thousand and seventy-six (98.2%) included a sample 
size estimation, 714 (65.2%) reported that participants, 
personnel and/ or outcome assessors were blinded, and 
770 ( 70.3%) were two-armed (table 3). The reporting of 
control group varied across interventions. For example, 
none of the trials of tuina or qigong reported the use of a 
placebo/sham control, while a sham control was used in 
almost half of the acupuncture trials (44.8%). Other com- 
monly used controls were western medicine (214, 19.5%), 
no intervention (181, 16.5%), non-pharmaceutical inter- 
ventions (133, 12.1%), acupuncture (70, 6.4%) and 
Chinese herbal medicine (52, 4.7%). In addition, our data 
extraction of TCM syndrome differentiation showed that 
65 of 290 (22.4%) trials on Chinese herbal medicine uti- 
lised syndrome differentiation either in the recruitment of 
participants (54 trials) or in the prescription of the herbal 
formula (11 trials). All these trials except one from 
Australia were registered by institutions in mainland 
China. 

The information about randomisation procedures, 
including the generation of allocation sequence or con- 
cealment, was under-registered and under-reported 
across all intervention types. The estimated sample size 
of the trials ranged from less than 10 (mostly pilot/feasi- 
bility trials) to over 1000 participants, with the majority 
(88.3%) of sample sizes between 20 and 500/trial (see 
online supplement 02). 

Five hundred and five (46.1%) of the registered trials 
indicated their status as 'completed'. Nineteen of these 
505 trials (3.8%) had protocol publications. Our 
searches in PubMed and the Chinese databases identi- 
fied full paper publications for 115 from the 505 com- 
pleted trials (22.8%). The 115 trials produced 136 
publications reporting study outcomes, among which 
123 publications were in English and 13 in Chinese. 
Among the 115 trials with publications, 53.9% (62/115) 
trials specified primary outcomes (table 4). When com- 
paring the 115 registered trials with their 136 publica- 
tions, inconsistency was identified in sample size 
estimation (11%), outcome assessor blinding (37.5%), 
secondary outcome (34.4%) and safety reporting 
(28.6%; table 4). Selective outcome reporting was found 
in 29.2% (19/65) of the subsequent publications when 
comparing the stated primary outcomes in the registries. 
Incomplete outcome data were addressed in 34 trial 
publications (25%). Of these, 80.9% (110/136) of the 
trial publications reported positive results. 
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Table 2 Conditions based on ICD-10 classification treated by Traditional Chinese Medicine 

Disease/conditions (ICD-10 codes) Number of RCTs Proportion (%) 



Pain 


146 


13.3 


M00-M99 Diseases of the musculoskeletal system and connective tissue 


128 


11.7 


G00-G99 Diseases of the nervous system 


95 


8.7 


K00-K93 Diseases of the digestive system 


78 


7.1 


I00-I99 Diseases of the circulatory system 


71 


6.5 


J00-J99 Diseases of the respiratory system 


69 


6.3 


F00-F99 Mental and behavioural disorders 


68 


6.2 


C00-D48 Neoplasms 


66 


6.0 


E00-E90 Endocrine, nutritional and metabolic diseases 


63 


5.7 


000-099 Pregnancy, childbirth and the puerperium 


50 


4.6 


Z00-Z99 Factors influencing health status and contact with health services 


41 


3.7 


Side effects of chemotherapy for cancer 


34 


3.1 


R00-R99 Symptoms, signs and abnormal clinical and laboratory findings, not 


32 


2.9 


elsewhere classified 






N00-N99 Diseases of the genitourinary system 


32 


2.9 


S00-T98 Injury, poisoning and certain other consequences of external causes 


28 


2.6 


I 00-I QQ Di^pa^p^ of thp ^kin and ^nhpiitanpnii*? ti^^np 


23 


2.1 


A00-B99 Certain infectious and parasitic diseases 


22 


2.0 


D50-D89 Diseases of the blood and blood-forming organs and certain disorders 


17 


1.6 


involving the immune mechanism 






H00-H59 Diseases of the eye and adnexa 


15 


1.4 


P00-P96 Certain conditions originating in the perinatal period 


9 


0.8 


H60-H95 Diseases of the ear and mastoid process 


8 


0.7 


Q00-Q99 Congenital malformations, deformations and chromosomal abnormalities 


1 


0.09 


Total 


1096 


100 



ICD, International Classification of Diseases; RCTs: randomised controlled trials. 



Among the 115 trials reporting outcomes, six trials 
showed other deviations from their original protocols. 
For example, one study was registered as prospective 
cohort study, but published as randomised trial; one trial 
was registered as parallel group, but published as a cross- 
over trial; one trial used different intervention from the 
registered record; and three trials showed inconsistent 
intervention arms from the registered information. 



To understand the adequacy of the trial registration, we 
compared the date of publications with the date of regis- 
tration and we found that 11 (9.6%) trials were registered 
later than the publication date, suggesting retrospective 
registration. Furthermore, in 41 trials (35.7%), the com- 
pletion date for the trials was earlier than the approval 
date of the registration, also suggesting inappropriate 
registration. 



Table 3 Methodological information of 1096 RCTs on Traditional Chinese Medicine from International Trial Registries 

Acupuncture/ 





CHM N=290 


acupressure 


Qigong/Taichi 


Tuina N=65 


Other therapies 




Itemsl 


(%) 


N=509 (%) 


N=66 (%) 


(%) 


N=166(%) 


Total 


Design 














Parallel 


269 (92.8) 


477 (93.7) 


62 (93.9) 


54 (83.1) 


162 (97.6) 


1024 


Crossover 


17 (5.9) 


24 (4.7) 


3 (4.6) 


9 (13.9) 


2(1.2) 


55 


Phase ll/lll* 


120/172 (69.8) 


141/186 (75.8) 


20/24 (83.3) 


21/24 (87.5) 


30/68 (44.1) 


332 


Placebo/sham-controlled 


194 (66.9) 


228 (44.8) 


0 (0.00) 


0 (0.00) 


41 (24.7) 


463 


Sample size estimation 


287 (99.0) 


496 (97.5) 


64 (97.0) 


65 (100.0) 


164 (98.8) 


1076 


Generation of allocation 


114 (39.3) 


63(12.4) 


8 (12.1) 


8 (12.3) 


85 (51.2) 


278 


sequence 














Allocation concealment 


47 (16.2) 


34 (6.7) 


4(6.1) 


8 (12.3) 


11 (6.6) 


104 


Blinding 


219 (75.5) 


346 (68.0) 


40 (60.6) 


38 (58.5) 


71 (42.8) 


714 


TCM syndrome differentiation 


65 (22.4)f 













*Only 474 trials (43.2%) provided information about phase. 

Blinding included participants, personnel and/or outcome assessors blinded. 

fTCM syndrome differentiation (bian zheng lun zhi): data limited to CHM. 

CHM: Chinese herbal medicine; RCTs, randomised clinical trials; TCM, Traditional Chinese Medicine. 
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Table 4 Comparison of methodological components between registered records and subsequent publications of RCTs 



Items 


rtecoras n— i id \ /oj 


rUDMCdiion ri— i oo \/o) 


inconbibiency relic \/o) 


Sample size 


114 (99.1%) 


136 (100%) 


17.6% 


Generation of allocation sequence 


3 (2.6%) 


80 (58.8%) 


66.7% 


Allocation concealment 


0 (0.0%) 


61 (44.9%) 


NA 


Participant blinded 


43 (37.4%) 


66 (48.5%) 


17.7% 


Practitioner blinded 


18 (15.7%) 


31 (22.8%) 


11.1% 


Outcome assessor blinded 


30 (26.1%) 


54 (39.7%) 


37.5% 


Primary outcome 


62 (53.9%) 


114 (83.8%) 


29.2% 


Secondary outcome 


52 (45.2%) 


96 (70.6%) 


34.4% 


Safety 


28 (24.4%) 


58 (42.7%) 


28.6% 


NA: not available; RCTs, randomised clinical trials. 



DISCUSSION 

The aim of prospective registration of clinical trials is to 
create transparency about the research methodology, 
and allow access to the trial protocol before the comple- 
tion of the trial. Researchers and systematic reviewers can 
then compare the registered information or published 
protocols and their subsequent publications to assess 
publication bias or selective outcome reporting. 13 14 

We observed a significant increase in the number of 
registered TCM trials in international clinical trial regis- 
tries over the past decade. Our evaluation of trial quality 
covers a wide variety of TCM interventions from acupunc- 
ture and Chinese herbal medicine to some very ancient 
interventions such as cupping and gua sha therapy. 
These studies mainly address chronic and long-term con- 
ditions such as pain, musculoskeletal and neurological 
problems. TCM trial design in terms of sample size esti- 
mation, use of placebo-control and blinding and the def- 
inition of primary and secondary outcome measures is 
improving according to the information held in the regis- 
ters. 15 However, we found that the generation of alloca- 
tion sequence and concealment were under-reported in 
the registered trials. This might be because these two 
items were not mandatory for the majority of registries. 
Interestingly, among 291 trials on Chinese herbal medi- 
cine, less than a quarter of the trials (n=65) utilised syn- 
drome differentiation, which is considered vital to the 
TCM diagnosis as the basis for classifying subtypes of par- 
ticipants and for the prescription of herbal formula. 
Almost all of these trials with syndrome differentiation 
were registered by institutions in China, which suggest 
that Chinese researchers pay more attention to the selec- 
tion of optimal participants and tailored treatment based 
on different TCM syndromes. 

We found that 29% of registered TCM trials showed 
selective outcome-reporting bias (discrepancies between 
the outcomes registered and the outcomes published). 
The inconsistency between the trial protocols and subse- 
quent publications also relates to sample size, blinding, 
primary and secondary outcomes as well as safety. This 
implies that deviations from protocols might be due to 
the study findings which then resulted in selective 
outcome reporting. A previous study demonstrated that 
31% of trials in Western medicine showed discrepancies 



between the registered protocols and the outcomes pub- 
lished in high-impact general medical journals, 13 which 
suggest that TCM researchers are not alone in reporting 
outcomes selectively. Our data also suggest that there 
might be inappropriate registration in about half of the 
trials. Accordingly, there is still ample space to improve 
the quality of trial registration. 16 Clinicians and policy- 
makers need unbiased results from clinical trials to 
make informed clinical decisions. 

Our study has some limitations. First, due to the lack 
of standardisation of the items required for registration 
in different registries some important information, such 
as randomisation methods, may be under-reported. 
Second, we only searched for publications for those 
trials indicating a 'completed' status. We do not know 
how often this information is updated in different regis- 
tries. Therefore, there may be trials in the registries that 
are not listed as 'completed', but that have been com- 
pleted. This may cause bias, particularly if the investiga- 
tors who update the status of their trials on the registry 
are also more likely to adhere to the methods described 
in the registry when writing up their results in the subse- 
quent publications. Third, as we undertook searches in 
PubMed and three Chinese databases only, we may have 
missed some studies that have been reported in other 
databases. In addition, there is a lag time between com- 
pleting a study and writing for publication, submission 
and peer review, all of which can be considerable. 
Fourth, we only looked at registered trials. A very large 
number of TCM trials are conducted without being 
registered, and here we can say nothing about their risks 
of bias and risks of random errors. In all likelihood, 
these may be even worse than those we have observed. 

We conclude that the study design and the quality of 
reporting of TCM trials have improved through pro- 
spective international trial registration compared with 
previous methodological studies, 1-6 although there are 
some inconsistencies between the registered trial proto- 
cols and subsequent publications and insufficient report- 
ing on syndrome differentiation. Publication bias as a 
consequence of selective outcome reporting is still wide- 
spread and similar to conventional medicine, therefore 
study conclusions should be interpreted with caution. In 
herbal medicine trials, it would be inappropriate if a 
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trial design does not utilise syndrome differentiation, 
and participants may not be properly treated. 
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